A Unified Factorization Algorithm for Points, Line Segments and Planes with Uncertainty Models

In this paper we present a uni ed factorization algorithm for recovering structure and motion from image sequences by using point features, line segments and planes. This new formulation is based on directional uncertainty model for features. Points and line segments are both described by the same probabilistic models and so can be recovered in the same way. Prior information on the coplanarity of features is shown to t naturally into the new factorization formulation and provides additional constraints for the shape recovery. This formulation leads to a weighted least squares motion and shape recovery problem which is solved by an e cient quasi-linear algorithm. The statistical uncertainty model also enables us to recover uncertainty estimates for the reconstructed three dimensional feature locations.